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METHOD AND APPARATUS FOR JOINT EQUALIZATION AND DECODING 
OF MULTIDIMENSIONAL CODES TRANSMITTED OVER 
MULTIPLE SYMBOL DURATIONS 

5 Field of the Invention 

The present invention relates generally to channel equalization and decoding 
techniques, and more particularly, to methods and apparatus for performing joint equalization 
and decoding of a multidimensional code. 

10 Background of the Invention 

Ethernet has become a successful media interface for local area networks (LANs) 
and has seen an exponential growth in achievable data rates over the past decade. In 1999, the 
IEEE 802.3ab task force specified the first Gigabit Ethernet over copper standard, lOOOBASE-T, 
O which allows for 1 Gb/s communications over four wire pairs of Category 5 unshielded twisted 
Pll pair (UTP) copper cabling for distances up to 100 m. However, due to the channel 
characteristics of Category 5 cabling and due to the fact that full duplex transmission is employed 
^ on each wire pair, a lOOOBASE-T receiver corresponding to one wire pair must cope with a 
w number of impairments, including intersymbol interference (ISI) caused by wire attenuation, 
m echo from its own transmitter, near end crosstalk (NEXT) from the adjacent three local 
transmitters, far end crosstalk (FEXT) from the remote transmitters of the three adjacent wire 
i;3 pairs, and noise from other sources, 

lOOOBASE-T Gigabit Ethernet employs 4-dimensional trellis-coded modulation 
(4D-TCM) to improve the noise margin. For a more detailed discussion of 4D-TCM encoding, 
see, for example, G. Ungerboeck, 'Trellis-Coded Modulation With Redundant Signal Sets, Parts 
25 I and U," IEEE Conamun. Mag., vol 25, pp. 5-21 (Feb. 1987). Whereas FEXT does not need to 
be cancelled in a lOOOBASE-T implementation, the noise from ISI, echo and NEXT impairments 
must be removed to achieve a target bit error rate of at least 10"^^ . In a digital signal processor 
(DSP) based VLSI implementation, the number of filter taps needed for the respective cancellers 
is very high leading to a VLSI implementation with significant power consumption. 
30 Therefore, a Gigabit Ethemet scheme employing four pairs of Category 6 UTP 

cabling was proposed in K. Azadet et al., "A Gigabit Transceiver Chipset for UTP CAT-6 Cables 

-1- 



Azadet 15-7 



in Digital CMOS Technology " IEEE Int. SoUd-States Circuits Conf. gSSCC) Dig. Tech. Papers, 
San Francisco, CA, 306-307 (Feb. 2000), that leads to a transceiver implementation with 
significantly lower hardware complexity. Category 6 UTP cabling has significantly better 
channel characteristics than Category 5 UTP cabhng, as it has a bandwidth of 250 MHz 
(Category 5: 100 MHz) and as it suffers less from NEXT. 

The transmission scheme for Category 6 Gigabit Ethernet is shown in FIG. 1. 
Each of the four wire pairs transmits data at a rate of 500 Mb/s only in one direction, and two 
wire pairs are used to achieve an overall data rate of 1 Gb/s for distances up to 100 m. Thus, 
there is no echo, and there are only two NEXT sources (for lOOOBASE-T there are three NEXT 
sources) and 1 FEXT source (for lOOOBASE-T there are 3 FEXT sources) on a single wire pair. 
For Category 6 UTP cabling, the impairments of NEXT and FEXT are so weak that they do not 
need to be cancelled to achieve a target bit error rate of at least 10"" . Also, the postcursor ISI 
after feedforward equalization is significantly shorter for Category 6 than for Category 5 cabling. 
Therefore, only ISI must be cancelled in Category 6 Gigabit Ethernet using adaptive equaUzation, 
leading to a VLSI implementation with significantly lower hardware complexity and power 
consumption than the lOOOBASE-T standard. 

As there is significant postcursor ISI in Category 6 Gigabit Ethernet, 4D-TCM 
must be employed as well to increase the noise margin. To benefit fix)m the potential coding gain 
of this coding scheme, joint equalization and trellis decoding must be performed. However, an 
optimum maximum likelihood sequence estimation (MLSE) detector would be far too complex, 
and the conventional suboptimum reduced-state sequence estimation (RSSE) algorithm cannot be 
applied, as the number of channels used for data transmission is smaller than the number of 
dimensions of the 4D-TCM code. In other words, under a 4D-TCM encoding scheme, two 
symbols corresponding to two dimensions of the four dimensional code are transmitted 
successively over the same wire pair. A need therefore exists for a joint postcursor equalizer and 
trellis decoder for such multidimensional codes, such as the treUis-coded Category 6 Gigabit 
Ethernet scheme. A fiorther need exists for an RSSE scheme that cancels the intrasymbol 
interference caused by other symbol components within the same multidimensional code symbol, 
as well as the intersymbol interference caused by previously transmitted multidimensional code 
symbols. 
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Summary of the Invention 

Generally, a method and apparatus are disclosed for performing joint equalization 
and decoding of multidimensional codes transmitted over multiple one-dimensional symbol 
durations. According to one aspect of the invention, an RSSE scheme is disclosed that cancels 
5 the intrasymbol interference caused by other symbol components within the same 
multidimensional code symbol. The disclosed RSSE technique for multidimensional codes 
applies where the number of trellis code dimensions is larger than the number of channels. 

The disclosed RSSE decoder computes the intersymbol interference caused by 
previously decoded multidimensional code symbols and subtracts the intersymbol interference 
10 from the received signal. In addition, the intrasymbol interference caused by other symbol 
components within the same multidimensional code symbol are computed and subtracted from 
^ the received signal. The disclosed RSSE decoder for an exemplary 4D-TCM code includes 2D 
branch metric units (2D-BMU) that calculate the 2D branch metrics for the two wire pairs based 

fy on the received 2D signals. A 4D-BMU combines these 2D branch metrics to calculate the 4D 

FU 

fB branch metrics corresponding to the transitions in the trellis diagram for 4D-TCM. The best 

m 

surviving paths into the eight code states are determined in an add-compare-select unit (ACSU) 
and then stored in the survivor memory unit (SMU). The 4D-BMU, ACSU and SMU can be 
f ij implemented in accordance with known techniques. 

l^s^ According to one aspect of the invention, the 2D-BMUs compensate for 

% intrasymbol interference caused by other symbol components within the same multidimensional 

code symbol. In addition, the DFU processes the survivor symbols from the SMU to calculate 

the intersymbol interference estimates for the trellis states and chaxmels, which are used by the 

2D-BMUS to calculate the 2D branch metrics. 

A more complete understanding of the present invention, as well as further 
25 features and advantages of the present invention, will be obtained by reference to the following 

detailed description and drawings. 

Brief Description of the Drawings 

FIG. 1 illustrates the transmission scheme for Category 6 Gigabit Ethernet; 
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FIG. 2 illustrates the four-dimensional subset partitioning for the 4D-TCM 
scheme proposed for Category 6 Gigabit Ethernet; 

FIG. 3 illustrates the trellis corresponding to the 4D-TCM code of FIG. 2; 

FIG. 4 illustrates the equivalent discrete time channel model for Category 6 
Gigabit Ethernet; 

FIG. 5 is a schematic block diagram of an exemplary 4D RSSE architecture in 
accordance with the present invention; 

FIG. 6 is a schematic block diagram illustrating the calculation of the ISI 

estimates in the DFU of FIG. 5; and 

FIG. 7 is a schematic block diagram illustrating the computation of the 2D branch 

metrics in the 2D-BMU of HG. 5. 

Detailed Description 

The present invention provides a modified RSSE technique for multidimensional 
trelHs codes that cancels the intrasymbol interference caused by other symbol components within 
the same multidimensional code symbol. While the present invention is illustrated herein using 
an exemplary 4D-TCM encoding scheme, the present invention can be appUed to any 
multidimensional codes transmitted over multiple one-dimensional symbol durations, as would 
be apparent to a person of ordinary skill in the art. In other words, the disclosed RSSE technique 
for multidimensional codes appUes where the number of trellis code dimensions exceeds the 
number of available channels. It is noted that while the present invention is illustrated herein 
using a 4D-TCM code, the present invention can be applied to other multidimensional codes as 
well, as would be apparent to a person of ordinary skill in the art. 

A VLSI prototype of a Gigabit transceiver for Category 6 cabling comprising 
analog signal processing, clock recovery and digital equalization has been presented m K. Azadet 
et al., "A Gigabit Transceiver Chipset for UTP CAT-6 Cables in Digital CMOS Technology," 
IEEE Int. Solid-States Circuits Conf. (ISSCC) Dig. Tech. Papers, San Francisco, CA, 306-307 
(Feb. 2000). However, in the CMOS implementation described therein, channel coding was not 
employed. Therefore, an illustrative embodiment of the present invention suggests using the 
same 4D-TCM code that was specified for lOOOBASE-T Gigabit Ethernet in "Physical Layer 
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Parameters and Specifications for 1000 Mb/s Operation Over 4 Pairs of Category 5 Balanced 
Copper Cabling, Type lOOOBASE-T," IEEE Standard 802.3ab-1999 (1999), to increase the noise 
margin of Category 6 Gigabit Ethernet. The present invention provides a joint postcursor 
equalizer and trellis decoder for such a trellis-coded Category 6 Gigabit Ethernet scheme. 
Four-Dimensional Trellis-Coded Modulation (4D-TCM) 

The 4D trellis code specified in the lOOOBASE-T standard for Category 5 Gigabit 
Ethernet can be applied to Category 6 Gigabit Ethernet as well. In contrast to lOOOBASE-T, 
however, where the ID symbols corresponding to the single dimensions of the 4D code are 
transmitted over four different wire pairs, two symbols corresponding to two dimensions of the 
4D code are transmitted successively over the same wire pair in the proposed trellis-coded 
Category 6 Gigabit Ethernet scheme. This is necessary, because only two wire pairs are used for 
data transmission in one direction while a 4D trellis code is being employed. 

The proposed Category 6 Gigabit Ethernet scheme employs pulse amplitude 
modulation with the five levels {-2, -1, 0, 1, 2} (PAM5) as the signaling scheme on each wire 
pair. The exemplary ID symbol rate is 250 Mbaud, and each PAM5 symbol carries 2 information 
bits. For coding purposes, the PAM5 symbol constellation is divided into the two ID subsets A = 
{-1, 1} and B= {-2, 0, 2}. The minimum Euclidean distance between ID subset A and B is 1. 
However, the Euclidean distance among symbols of the same ID subset is = 4 as shown in 
HG. 2. 

As shown in FIG. 2, by grouping different combinations of four ID subsets, which 
are transmitted successively over two wire pairs, the eight 4D subsets SO, SI, ... S7 are formed. 
Two dimensions of the 4D symbol are transmitted over the first wire pair, and the other two 
dimensions are transmitted over the second wire pair used for transmission in the same direction. 
The exemplary 4D symbol rate is 125 Mbaud. Each 4D symbol carries eight information bits. 
The 4D subset partitioning shown in FIG. 2 guarantees a minimum EucUdean distance of A^ = 4 
between different 4D symbols of the same 4D subset and a minimum distance of A^ = 2 
between 4D symbols of different even subsets (SO, S2, S4, S6) or odd subsets (SI, S3, S5, S7). 

The trellis corresponding to this 4D-TCM code is shown in FIG. 3. There are 
eight states, and four branches leave or enter each state. Each transition in the treUis corresponds 
to a 4D subset as specified in HG. 2. Due to the subset partitioning and labeling of the 



Azadet 15-7 



transitions in the code trellis, the minimum Euclidean distance between allowed sequences is 
= 4, which corresponds to an asymptotic coding gain of 101og4 = 6 dB over uncoded 4D 
PAM5 in an ISI free channel with additive white Gaussian noise (AWGN). However, for a fair 
comparison, the performance gain of the described 4D-TCM code over uncoded 4D PAM4 with 
the ID symbol levels {-1.5, -0.5, 0.5, 1.5} has to be considered, which also achieves a throughput 
of 1 Gb/s (2 information bits per PAM4 symbol) at a ID symbol rate of 250 Mbaud. The 
performance loss due to the ID symbol alphabet expansion from PAM4 to PAM5 is 3 dB under 
the assumption that all ID PAM4 and PAM5 symbols are equally likely. However, the 4D code 
symbols of tiie exemplary 4D-TCM code set have been chosen such that 4D PAM5 code 
symbols, which are close to the origin in the 4D symbol constellation, are more likely than outer 
4D code symbols. Considering this shaping of tiie 4D PAM5 code symbol constellation, tiie 
performance penalty due to the expansion of tiie symbol constellation reduces to approximately 
1.8 dB. This means that the asymptotic, ISI-free coding gain of the discussed 4D-TCM code over 
uncoded 4D PAM4 is approximately 4.2 dB. 

System Model 

As in lOOOBASE-T Gigabit Etfiemet, a Category 6 Gigabit Etiiemet receiver 
would employ a feedforward equalizer for each wire pair after tiie analog-to-digital conversion to 
make tfie channel minimum-phase and whiten tiie noise. Then, tiie only impairments, which 
must be taken care of, are postcursor ISI and AWGN. After feedforward equalization, the 
postcursor ISI typically spans eight ID symbol periods, i.e. the postcursor channel memory is 
L = 8. 

The equivalent discrete time model after feedforward equalization for tiie overall 
transmission system corresponding to a particular data direction is shown in HG. 4. Information 
symbols carrying eight information bits are fed into tiie TCM encoder (410 and 420) at 
125 MHz, where m is the time index corresponding to tiie 4D symbol rate of 125 Mbaud. The 
rate 2/3 convolutional encoder 410 operates on two information bits to produce tiiree coded bits, 
which are used to select one of the eight 4D subsets, while tiie otiier uncoded bits are used to 
select one 4D symbol {a„,a„^i,KA^\) in tiie symbol mapper 420 witiiin tiie chosen 4D subset, 
where the time index n corresponds to the ID symbol rate of 250 Mbaud. The relationship 
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between w and n is that m is the closest integer smaller than n/2: m = [n/2j. The 2D 
component a„ = i,a„,a„^{) of the 4D symbol is transmitted over the first wire pair, and the other 
2D component ^„=(^,fe„+i) over the second wire pair. The channel outputs at time n 
corresponding to the two wire pairs are given by 

where {/ } and {g, } at stages 430 and 440, respectively, are the coefficients of the equivalent 
discrete-time channels corresponding to the two wire pairs. v„ and w„ are the corresponding 

whitened noise sequences. 

The sequence estimator 450 processes the received 4D symbols (r„,r„^i,5„,s„+i) 

□ to output an estimate for the information symbol 7„ . The optimum method of detecting a 
ru trellis-coded data sequence in the presence of ISI and AWGN is maximum likelihood sequence 
|;5 estimation (MLSE), which applies the Viterbi algorithm to the super trelUs defined by the 

I; J} if 

in concatenation of the trellis coder and the two channels with memory. However, MLSE would be 
lis prohibitively expensive to implement for Category 6 Gigabit Ethernet due to the large 4D symbol 
constellation and channel memory. The number of states in the super treUis would be 
^'0 S X 2*^'^ = 8 X 2*'^* = 3.4jc10*°, where S is the number of code states and 5 the number 

1^* of information bits per 4D symbol. 

However, it is not necessary to process all these states to benefit from the coding 

20 gain offered by the 4D-TCM code. Most of the gain can be achieved by a sequence estimation 
method with reduced complexity. It has been shown that compared to other suboptimum joint 
equalization and decoding algorithms, the reduced-state sequence estimation (RSSE) algorithm 
or variants of it are a good trade-off between error rate performance, hardware complexity and 
maximum throughput in high-speed applications such as Gigabit Ethernet over copper. 

25 Multidimensional RSSE 

RSSE reduces the hardware complexity of MLSE by processing only a limited 
number of states. In tiie following, tiie simplest form of RSSE is treated where the reduced-state 
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trellis is equal to the code trellis and postcursor ISI is cancelled for each code state separately by 
taking ttie symbols from tiie corresponding survivor path as tentative decisions. In this case, 
RSSE simplifies to decision-feedback sequence estimation (DSFE). The modified 
multidimensional RSSE technique employed herein can also be applied when more general 
RSSE algorithms are used, as would be apparent to a person of ordinary skill in the art. 

In the past, RSSE has usually been appUed for joint equaUzation and decoding of 
ID or 2D trelUs codes. RSSE can also be appUed to the 4D trellis code specified in lOOOBASE- 
T without difficulty, as the 4D code symbols are transmitted over four wire pairs, i.e., each 
dimension of the trellis code has its own channel. However, in the proposed trellis-coded 
Category 6 Gigabit Ethernet scheme, two dimensions of the specified 4D trellis code are 
transmitted over the same channel successively, causing problems to apply RSSE in its original 
form. Specifically, RSSE as described, for example, in M. V. Eyuboglu and S. U. Qureshi, 
"Reduced-State Sequence Estimation For Coded Modulation On Intersymbol Interference 
Channels," IEEE J. Select. Areas Commun., vol. 7, 989-995 (Aug. 1989) or P. R. Chevillat and 
E. Eleftheriou, "Decoding of Trellis-Encoded Signals in the Presence of Intersymbol Interference 
and Noise," IEEE Trans. Commun., vol. 37, 669-676 (Jul. 1989) only cancels the ISI caused by 
previously transmitted multidimensional code symbols, but is not able to cancel the intrasymbol 
interference (InSI), which is caused by other ID symbol components within the same 
multidimensional code symbol. 

In the case of Category 6 Gigabit Ethernet, the received 2D signal component 
corresponding to one wire pair, e.g., r„ =(r„,r„^i) does not only contain ISI from past 2D 
symbol components a„_j =(a„_^j,a„_^j,,), 1< j < \LJ2\ transmitted over this wire pair, but 
the odd ID signal component r„^i of r„ contains InSI caused by the even ID symbol component 
a„ of fl„ = (a„,a„+,) • RSSE in its original form cannot calculate separate InSI estimates for all 
states at time m as needed, as survivor symbols corresponding to the transmitted ID signal fl„ are 
not available in the survivor history yet, but become only available at time m + l. Therefore, the 
present invention provides a modified 4D RSSE structure for Category 6 Gigabit Ethernet as 
shown in FIGS. 5-7, which can be easily generalized to other multidimensional trellis codes. The 
proposed 4D RSSE architecture operates at an exemplary clock rate of 125 MHz. 
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FIG. 5 is a schematic block diagram of an exemplary 4D RSSE decoder 500 in 
accordance with the present invention. The 2D branch metric units (2D-BMU) calculate the 2D 
branch metrics for the two wire pairs based on the received 2D signals and . The 4D- 

BMU combines these 2D branch metrics to calculate the 4D branch metrics corresponding to the 
5 transitions in the trellis diagram of FIG. 3. The best surviving paths into each of the eight code 
states are determined in the add-compare-select unit (ACSU) and then stored in the survivor 
memory unit (SMU). The decision-feedback unit (DFU) takes the survivor symbols from the 
SMU to calculate the ISI estimates for all eight code states and two wire pairs, which are used by 
the 2D-BMUs to calculate the 2D branch metrics. 
10 The 4D RSSE decoder 500 shown in FIG. 5 differs from a conventional 

implementation of a RSSE decoder for the lOOOBASE-T standard in terms of the implementation 
of the DFU and 2D-BMU, which are explained hereinafter. The 4D-BMU, ACSU and SMU can 
□ be implemented, for example, using the lOOOBASE-T RSSE decoder described in E.F. Haratsch 
}fj and K. Azadet, "A 1-Gb/s Joint Equalizer and Trellis Decoder for lOOOBASE-T Gigabit 
% Ethernet," IEEE J. SoUd-State Circuits, vol. 36, 374-384 (Mar. 2001) or United States Patent 
iJI Application Serial Number 09/471,920, filed Dec, 23, 1999, entitled "Method and Apparatus for 
^ Shortening the Critical Path of Reduced Complexity Sequence Estimation Techniques," each 
\ ^ incorporated by reference herein. 

i;9 Calculation of ISI Estimates 

FIG. 6 is a schematic block diagram illustrating the calculation of the ISI 
estimates for the trellis code state and the first wire pair at time m in the DFU of FIG. 5. This 
block is required 8x2 = 16 times, as there are eight code states and two wire pairs. The ISI 
estimate c^(p^) corresponding to the even received ID signal is calculated using the survivor 

symbols {L^j{pJ}^{ia^,2^iPjA-ij^^^^ 1^7^ which correspond to and 

25 are available from the SMU of FIG. 5: 

CniPm) - XJ^^nM ^ (3) 

The ISI estimate c^^iip^) corresponding to the odd received ID signal r„+j is 
calculated accordingly: 
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C(P«) = Em /M«n-. (P«) (4) 

However, as previously indicated, c^^i(p„)does not consider the InSI contained 
in r„^., caused by the ID data symbol fl„ . A survivor symbol corresponding to a„ is not 
available in the SMU yet. In the RSSE structure of the present invention, the InSI is cancelled in 
the 2D-BMU, as discussed hereinafter. 
Calculation of 2D Branch Metrics 

HG. 7 is a schematic block diagram illustrating ttie computation of the 2D branch 
metrics for the first wire pair and state p„ in the 2D-BMU of HG. 5. This block is required 
8x2 = 16 times, as there are eight code states and two wire pairs. First, the ISI estimates 
calculated according to equations (3) and (4) are used to calculate ISI-free estimates jc„ (p„) and 

K^iiPj for fn and r„^,: 

x„ipj = r„-c„ipj, (5) 

Then ;c„(p„) is sliced to the closest A-type symbol x^(pj and 5-type 
symbol x„^(p„), respectively. The ID Euclidean distances between x„ip„) and the respective 
sliced signal points are calculated according to 

eW„{p„,A) = (x„ipJ-x^(pJ)\ (7) 

elD„(p„,B) = MpJ-x!ipJ)\ (8) 

^«(Pm) is the A-type data estimate for the transmitted symbol a„, which 
corresponds to state p„ . Therefore, an ISI and ItiSI ftee estimate a;„^i(P„,A) for r„^i under the 
assumption that x^ (p„) is the data estimate for a„ is calculated as 

^„.i(P«.^) = <.i(Pj-/i^«(Pj- (9) 

Accordingly, assuming that x„"ip„) is the data estimate for a„ , the ISI and InSI 
free estimate x„^i(p„,B) for r„^i is given by 

^„.i(P«.5) = ^Li(Pj-/i^«(Pj- (10) 
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After slicing (p„ , A) and (p„ , B) to the closest A-type and B-type PAM5 
symbols x^^i(p„,A), x^^,ip„,A), xi^(p„,B) and 5), respectively, the corresponding 

Euclidean distance metrics are calculated as follows: 

5 eW„,,ip„,AB) = {x„,,ip„,A)-xl,(p„,A))\ (12) 

elD„^,(p„,BA) = (x„,,(p„,B)-xt,(p„,B))\ (13) 

eW„,,{p„,BB) = {x„,,ip^,B)-xl,{p„,B)f . (14) 

All four possible 2D branch metrics corresponding to the first wire pair and state 
p„ are computed by combining the respective ID Euclidean distance metrics: 

10 e2D„ {p„ , AA) = elD„ (p^ , A) + eW„^, (p„ ,AA), (15) 

□ e2D„ (p„ , AB) = elD„ (p„ , A) + elD„^, (p„ , AB) , (16) 

ru e2P„(p„,BA) = dD,(p„,B) + elD„,i(p„,BA), (17) 

S e2D^ip^,BB)^elD^(p^,B)^elD^,,(p^,BB). (18) 

m These 2D branch metrics are then combined with the 2D branch metrics of the 

y second wire pair which are calculated in the same manner as described above to compute the 4D 

pes 

I ^" branch metrics corresponding to the trellis transitions from state in the 4D-BMU. 

It is to be understood that the embodiments and variations shown and described 
jjait; herein are merely illustrative of the principles of this invention and that various modifications 
may be implemented by those skilled in the art without departing from the scope and spirit of the 
20 invention. 
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